Combining Relevance Assignment with Quality of the Evidence to Support Guideline Development

Clinical practice guidelines are used to disseminate best practice to clinicians. Successful guidelines depend on literature that is both relevant to the questions posed and based on high quality research in accordance with evidence-based medicine. Meeting these standards requires extensive manual review. We describe a system that combines symbolic semantic processing with a statistical method for selecting both relevant and high quality studies. We focused on a cardiovascular risk factor guideline, and the overall performance of the system was 56% recall, 91% precision (F0.5-score 0.81). If quality of the evidence is not taken into account, performance drops to 62% recall, 79% precision (F0.5-score 0.75). We suggest that this system can potentially improve the efficiency of the literature review process in guideline development.